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1.  Introduction 

This  is  the  final  progress  report  for  ARO  Grant  DA  AG  19-00-1-0465  awarded  to  the  Depart¬ 
ment  of  Mathematics  of  Washington  State  University  for  the  period  July  24,  2000  through  July 
23,  2003.  This  grant  was  based  on  the  proposal  39877-MA  Low- Complexity  Interior  Point  Algo¬ 
rithms  for  Stochastic  Programming:  Derivation,  Analysis,  and  Performance  Evaluation  submitted 
to  ARO  by  the  author. 

This  report  is  structured  according  to  the  guidelines  stated  in  Reporting  Instructions  (ARO 
Form  18).  As  required  in  those  guidelines,  §4  of  this  report  contains  a  list  of  publications  and 
reports  that  acknowledge  support  from  this  grant,  and  §7  contains  a  bibliography  of  references. 
In  the  rest  of  this  report  we  indicate  references  to  items  in  this  report  with  item  number(s)  within 
double  brackets  and  references  to  items  in  the  bibliography  with  item  number  (s)  within  single 
brackets.  We  shall  refer  to  the  proposal  by  the  symbol  [P]. 


2.  Statement  of  the  problem  studied 

This  section  follows  the  proposal  [P]  closely.  The  project  was  concerned  with  new,  low- 
complexity  interior  point  algorithms  for  stochastic  programs  including  the  identification  of  variants 
that  are  efficient  in  practice.  We  take  1  as  our  model 

minimize  Z(x  i)  :=  qi(xi)  +  Q2(xi) 

subject  to  X\  €  Si  (1) 


where  x\  €  Rni  is  the  decision  variable,  and  the  feasible  set  S\  C  IT1  and  the  cost  function 
qi  :  Rni  -»  R  constitute  deterministic  data.  The  function  Q2  :  Rni  ->  R  is  defined  recursively  as 
follows:  Qt(xt- 1)  :=  £[Qi(xt_i,^t)]  is  the  expectation  with  respect  to  random  data  £t  €  R"‘  of 
the  value  of  a  function  Qt  for  t  =  2, 3, . . . ,  AT;  and  the  dependence  of  function  Qt  on  xt~i  and  a 
realization  of  is  specified  by 


Qt(xt-i,€t)  :=  inf  {qt(xt,£t)  +  Qt+\{xt)  ■  Xt  €  St(zt-i,£t)},  t  —  2,3, . . .  N  —  1,  and  (2) 

xt€Rn* 


Qn{xn~\,(,n)  ■=  inf  {qN{xN,(N)  ■  Xn  €  Sn(xn-1,€n)}, 

xjv€RnW 


(3) 


where  :  Rnt  R  and  St{xt-i,(t )  CRni,t  =  2,3,...,JV.  We  assume  that  random  data 

have  the  given  discrete  probability  distribution 


{(£U),i  =  l,2 


(4) 


for  t  =  2,3,...  ,N,  so  that  Qt{-)  :=  '£fL\plQt(-,Zt)- 

We  refer  to  the  problem  defined  by  (1-4)  as  a  multistage  stochastic  program  with  recourse. 
Such  an  optimization  problem  is  one  of  the  ways  in  which  the  following  multistage  stochastic 
decision-making  problem  illustrated  in  Figure  1  on  p.  3  (where  Kt  :=  n(=2Ar,  <  =  2,3,... ,  N) 
may  be  formulated  for  solution.  The  vector  x\  is  a  decision  that  has  to  be  made  at  present  (the 
stage  1).  Later,  at  stage  2,  a  realization  ^  of  second  stage  random  data  £2  becomes  available, 

1We  use  the  symbol  ‘:=’  to  indicate  equality  by  definition  in  mathematical  contexts,  and  assignment  in  statement  of 
algorithms.  Bold-face  letters  denote  random  variables  while  corresponding  normal- face  letters  denote  their  realizations. 
Superscript  T  denotes  transposition  of  vectors  and  matrices.  Other  superscripts  on  variables  often  denote  indices  rather  than 
powers. 
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and  a  recourse  decision  xl2  may  be  taken  if  necessary.  This  recourse  decision  is  chosen  from 
the  set  52(^1,^)  at  a  cost  9204,  £2)-  Note  that  the  set  S2  depends  on  the  decision  x\  already 
implemented  in  stage  1,  and  the  specific  realization  £l2  of  the  random  variable  £2  observed.  The 
decision  process  has  a  total  of  N  stages.  At  each  of  the  stages  2  through  N,  realizations  of  random 
variables  are  observed,  and  recourse  decisions  are  made  similarly.  The  formulation  (1-4)  states 
that  the  decision  x\  €  S\  that  needs  to  be  made  at  present  without  the  knowledge  of  realizations 
of  random  variables  £2,  £3,  •  •  • ,  £/v  that  will  be  observed  in  future  stages,  is  to  be  made  so  that  the 
cost  q\{x\)  plus  the  average  of  the  costs  of  future  recourse  decisions  £>2(^1)  is  minimized. 

There  are  many  applications  of  stochastic  programs  with  recourse.  These  application  ar¬ 
eas  include  energy  planning,  agriculture,  airline  scheduling,  optimal  control  problems,  industrial 
management,  natural  resource  management  such  as  lake  eutrophication  management  and  forestry 
management,  telecommunication,  and  problems  in  mathematical  finance  such  as  portfolio  opti¬ 
mization.  Please  consult  the  monographs  [8,  21,  9,  3]  and  references  contained  therein.  We  now 
mention  some  more  specialized  applications.  Midler  and  Wolmer  [15]  describe  stochastic  pro¬ 
gramming  models  for  optimally  scheduling  airlift  operations,  devoting  special  attention  to  airlift 
missions  among  airports  in  the  continental  U.S.,  aerial  ports  of  embarkation,  overseas  bases  and 
aerial  ports  of  debarkation.  Martel  and  Al-Nuaimi  [14]  describe  models  for  tactical  manpower 
planning.  A  two-stage  decision  model  for  weapon  acquisition  with  target  uncertainty  is  described 
by  Nickel  and  Mangel  [17].  Refer  to  Manne  and  Richels  [13]  for  an  application  related  U.S.  breeder 
reactor  program.  Cariiio,  Myers  and  Ziemba  [6]  describe  a  comprehensive  investment,  liability 
and  risk  planning  model  developed  by  Frank  Russell  Company  for  Yasuda  Fire  and  Marine  Insur¬ 
ance  Co.,  Ltd.  in  Japan.  Fragniere  and  Haurie  [10]  describe  a  model  to  identify  optimal  policies 
that  the  city  of  Geneva  may  adopt  for  meeting  CO2  emission  standards  required  by  the  Federal 
Government  of  Switzerland  by  the  year  2005. 

The  representation  (1-4)  is  mathematically  equivalent  to  the  large-scale  mathematical  program 


kt  kN 

minimize  Z=  giQci)  +  •  •  •  +  ]T]  q[{x\)  +  ...  +  ^9^0*^) 

1=1  i=i 

subject  to  2:1  €  Si, 

x[  €  Stixf^,^) 

1  =  1,2,... ,KU  t  =  2,3,...,N. 

in  (5),  kt  =  n \=2Kr, 

a(l,t)  :=  \l/Ktl 

an  +\  _  j  Kt  I*1  mod  Kt  =  0 

P[l'  ’  •“  \  l  mod  Kt  if  /  mod  Kt  ^  0, 

/  =  1, 2, . . . , Kt,  t  =  2,3,...,N, 

and  <7i(-)  :=  qi(-)  and  $(•)  :=  p[  ■  9<(-,^f(,,<))  for  /  =  1,2, . . .  ,Kt,  t  =  2, 3, . . . ,  N  where 
Pi  “  1. 

p[  ~  pf{t]  ■  p?(M),  /  =  1,2,..., At,  t  =  2,3, . . .  ,N. 


(5) 


This  equivalence  follows  from  the  decision  tree  representation  of  (1-4)  in  Figure  1.  In  Figure 
1,  a  path  from  the  root  to  a  leaf  is  termed  a  scenario.  The  number  of  scenarios  is  Km  — 
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n^-2-^r-  Assume  that  the  set  St  can  be  represented  using  mt  constraints.  Then  problem  (5)  has 
m  +  Ya=2  ntKt  variables  and  m\  +  YhtL 2  mt^t  constraints.  Typically,  even  when  the  dimensional 
parameters  mt,  fit  and  ut  have  moderate  values,  problem  (5)  is  large-scale  because  Kt  depends 
exponentially  on  vt  and  hence  Kt's  and  in  turn  Kt  s  are  large.  By  way  of  illustration  suppose 
that  N  :=  5,  and  that  vt  :=  10,  nt  :=  7  and  mt  :=  6  for  t  =  2, 3,4, 5.  Suppose  further  that 
each  of  the  vt  components  of  the  random  variable  has  only  two  possible  realizations,  so  that 
Kt  =  2"*  =  210  for  t  =  2,3, 4, 5.  It  follows  that  we  have  over  a  billion  scenarios,  and  that  (5)  has 
over  7  billion  variables  and  over  6  billion  constraints.  Note  that,  as  this  example  illustrates,  a 
good  indicator  of  the  large-scale  nature  of  a  stochastic  program  is  its  number  of  scenarios. 

As  the  above  example  implies,  practical  stochastic  programs  are  often  prohibitively  large 
for  their  solution  by  general-purpose  optimization  algorithms.  (See  for  example,  [16,  Table  V] 
where  this  is  seen  for  the  popular  general-purpose  code  LOQO.)  Fortunately,  stochastic  programs 
possess  structures  (note  for  example,  the  structure  in  the  constraint  matrix  in  (5)  if  the  sets  St 
could  be  represented  by  a  finite  number  of  linear  constraints)  that  may  be  exploited  to  advantage 
in  computational  algorithms.  Algorithms  for  stochastic  programs  must  invariably  exploit  the 
special  structures  present  in  them.  Despite  tremendous  advancements  in  the  development  of  such 
special-purpose  algorithms,  and  in  computer  hardware  and  software,  computational  solution  of 
practical  stochastic  programs  is  still  a  challenging  problem  [20,  16]. 

Research  during  the  past  decade  into  interior  point  methods  have  resulted  in  significant  ad¬ 
vancements  in  the  solution  of  deterministic  linear  programs.  These  advances  are  characterized 
by  the  identification  of  algorithms  with  low  polynomial  complexity  and  excellent  practical  perfor¬ 
mance.  Motivated  by  this  success  researchers  have  begun  to  explore  interior  point  methods  for 
other  areas  of  deterministic  mathematical  programming  such  as  convex  and  semidefinite  program¬ 
ming,  and  advances  similar  to  those  in  the  case  of  linear  programming  are  being  made.  However, 
surprisingly  very  little  research  has  been  devoted  to  interior  point  methods  for  stochastic  pro¬ 
gramming.  In  the  context  of  stochastic  programming  the  desirable  interior  point  methods  should 
again  have  low  complexity  and  good  practical  performance.  What  we  mean  by  low  complexity 
here  is  complexity  lower  than  that  of  a  general-purpose  interior  point  method  applied  on  the 
same  problem  in  terms  of  the  number  of  scenarios.  Indeed,  such  low  complexity  is  a  theoretical 
measure  of  how  well  the  structure  of  the  stochastic  program  has  been  exploited  by  the  relevant 
special-purpose  algorithm. 

Against  the  above  background,  the  broad  purpose  of  the  project  was  to  derive  interior  point 
methods  with  such  low  complexity  for  certain  important  special  cases  of  (1-4),  and  to  identify 
variants  with  good  practical  performance  via  careful  implementation  and  testing. 

We  conclude  this  introductory  section  by  indicating  certain  special  cases  of  (1-4)  investigated 
in  this  project.  A  two-stage  stochastic  linear  program  results  when  N  :=  2,  the  functions  q\  and 
<72('j  Ci)  are  linear,  and  the  sets  S\  and  S2  are  specified  by  linear  constraints.  A  two-stage  stochastic 
quadratic  program  (TSSQP)  is  similarly  defined  with  q\  and  q?(- ,  f)  being  quadratic  instead  of 
being  linear.  When  N  >  2,  the  functions  q\  and  qt{-,£t)  for  t  =  2,3, ...  ,N  are  linear,  and  the 
sets  St  for  t  =  1, 2, . . . ,  N  are  specified  by  linear  constraints  we  have  a  multistage  stochastic  linear 
program  (MSSLP).  A  multistage  stochastic  convex  program  (MSSCP)  is  similarly  defined  with  the 
functions  q\  and  qt{-,£t)  for  t  =  2, 3, . . . ,  N  being  convex  instead  of  being  linear. 
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3.  Summary  of  the  most  important  results 

In  this  section  we  summarize  the  most  important  results  of  this  project.  Whenever  possible, 
we  do  so  relative  to  the  tasks  indicated  in  [P,  pp.  D-14,15,16].  Please  note  that  we  present  these 
results  in  order  of  importance.  We  begin  with  results  obtained  through  work  performed  in  Tasks 
2,  3  and  4. 

(a)  Task2 

This  task  was  on  the  development  of  a  test  problem  collection  of  stochastic  programs. 

We  have  developed  such  a  test  problem  collection  consisting  of  instances  of  TSSLP’s  and 

MSSLP’s  with  the  following  specific  features. 

(i)  At  present  the  collection  involves  test  problems  from  11  application  areas:  airlift  op¬ 
erations  scheduling,  forest  planning,  electrical  investment  planning,  selecting  currency 
options,  financial  planning,  design  of  batch  chemical  plants,  energy  and  environmental 
planning,  network  models  of  asset  or  liability  management,  cargo  network  scheduling, 
telecommunication  network  planning,  and  bond  investment  planning. 

(ii)  The  102-page  technical  report  in  Item  [[4(e)(i)]]  below  describes  this  test  problem 
collection  in  detail. 

(iii)  We  have  created  a  web  site  (http://www.uwsp.edu/math/afelt/slptestset.html) 
providing  free  access  to  our  test  problem  collection.  For  each  of  the  11  families  in  the 
collection,  six  pieces  of  information  are  provided:  description  of  the  application  and 
problem  notation;  problem  statement  in  the  same  notation;  numerical  example  (when 
practical);  reconciliation  to  the  standard  notation  for  stochastic  programs  (as  given  in 
Item  [[4(a) (iii)]]  below);  SMPS  [4]  data  files  for  each  problem  instance;  and  optimal 
solution  for  each  problem  instance. 

This  web  site  is  maintained  by  my  former  doctoral  student  Andrew  J.  Felt,  who  is  now 
an  Assistant  Professor  of  Mathematics  and  Computing  at  University  of  Wisconsin, 
Stevens  Point. 

(iv)  We  presented  details  of  our  collection  to  the  research  community  at  the  Ninth  Interna¬ 
tional  Conference  on  Stochastic  Programming  held  in  Berlin,  Germany  during  August 
25-31,  2001.  (See  Item  [[4(c)(iii)]]  below.)  Our  work  was  well-received  as  fulfilling  a 
need  that  existed  in  the  field  for  a  long  time.  Researchers  at  the  meeting  expressed  the 
view  that  a  quality  test  set  would  not  only  assist  algorithm  development  by  providing 
a  standard  set  of  problems  with  which  to  challenge  new  algorithm  implementations, 
but  also  the  teaching  of  stochastic  programming. 

To  further  publicize  our  test  problem  collection  we  prepared  Item  [[4(a)  (iii)]]  which  has 
been  accepted  for  publication  in  INFORMS  Journal  on  Computing.  That  paper  also 
includes  a  request  for  information  on  additional  test  problems  from  researchers  to  be 
cataloged  in  a  standard  manner  and  made  available  via  our  web  site.  In  this  sense  this 
is  an  “evolving  effort” ,  and  our  web  site  will  be  updated  as  new  test  problem  instances 
are  received. 

(v)  The  addition  of  new  test  problems  from  novel  applications  to  our  test  problem  collection 
has  already  begun.  Martel  and  Al-Nuaimi  [14]  have  presented  a  two-stage  stochas¬ 
tic  programming  approach  for  optimal  tactical  manpower  planning.  Their  model  is 
based  on  certain  assumptions  that  may  be  relaxed  to  make  the  model  more  relevant 
to  applications.  Cristina  Cacho,  a  graduate  student  of  the  PI,  completed  a  Master’s 
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project  on  generalizing  the  model  in  [14]  in  May  2003.  Six  new  test  problem  instances 
based  on  the  resulting  extended  model  will  be  added  to  the  test  problem  collection 
(http://www.uwsp.edu/math/afelt/slptestset.html)  in  the  near  future.  See  Item 
[[4(c) (vii)]]  below. 

Yan  Zhang,  a  new  graduate  student  of  the  author,  has  just  begun  creating  similar 
models  for  electric  power  system  planning. 

(b)  Task  3 

The  purpose  of  Task  3  is  to  address  issues  necessary  to  implement  a  new  family  of  cutting 
plane  algorithms  (CPA)  for  two-stage  stochastic  linear  programs  (TSSLP).  In  a  previous 
paper  [2],  the  author  has  shown  that  certain  algorithms  in  this  family  of  CPA’s  have  poly¬ 
nomial  complexity  that  depends  linearly  on  the  number  of  realizations  K.  This  is  the  lowest 
complexity  known  for  any  computational  algorithm  for  TSSLP’s,  and  this  family  of  algo¬ 
rithms  was  the  first  to  achieve  this  complexity  bound.  Since  this  is  a  theoretical  upper  bound 
on  the  computational  work  required,  Task  3  was  to  implement  specific  algorithms  in  this 
class  so  that  using  the  test  problems  developed  in  Task  2,  their  practical  performance  could 
be  assessed.  We  have  performed  the  following  specific  work  on  this  task. 

(i)  While  the  SMPS  [4]  data  format  (see  Item  [[3(a)  (iii)]])  for  specifying  data  defining  the 
TSSLP’s  and  MSSLP’s  is  very  convenient  and  natural  for  users,  algorithms  are  defined 
using  the  data  as  specified  in  the  mathematical  form  of  equations  (1-4)  and  Figure  1. 
We  have  developed  a  data  structure  suitable  for  implementing  algorithms  for  TSSLP’s 
and  MSSLP’s,  and  written  C  codes  that  converts  data  in  the  SMPS  standard  format 
into  this  data  structure.  While  we  developed  this  data  structure  and  the  conversion 
routines  for  implementing  and  testing  our  algorithms,  this  has  been  done  in  a  very 
general  manner  so  that  any  algorithm  for  stochastic  programs  can  be  implemented 
similarly. 

Due  to  the  lack  of  routines  that  convert  SMPS  data  into  a  general  data  structure 
suitable  for  implementing  algorithms  for  TSSLP’s  and  MSSLP’s,  different  input  data 
formats  are  being  used  in  different  existing  implementations  of  algorithms  for  TSSLP’s 
and  MSSLP’s.  (Indeed,  we  observed  this  when  we  attempted  to  obtain  codes  of  existing 
algorithms  for  TSSLP’s  and  MSSLP’s  for  comparison  with  the  implementation  of  our 
new  algorithms.)  The  disadvantage  is  that  such  codes  cannot  directly  be  tested  on 
large  problem  instances  for  which  users  have  data  only  in  the  SMPS  format. 

We  believe  that  our  new  data  structure,  and  our  new  C  routines  for  converting  data 
in  SMPS  format  into  that  data  structure,  would  be  used  by  developers  of  software  for 
TSSLP’s  and  MSSLP’s.  This  would  promote  the  adoption  of  the  SMPS  standard  for 
specifying  input  for  software  for  stochastic  programs  allowing  the  full  realization  of  the 
benefits  of  the  SMPS  standard.  It  will  also  allow  convenient  testing  and  use  of  such 
software  on  large  meaningful  applications. 

As  an  examination  of  our  codes  would  demonstrate,  developing  the  data  structure  and 
the  conversion  routines  for  general  use  was  a  major  undertaking. 

(ii)  This  work  formed  the  Individual  Project — MS  in  Applied  Mathematics  of  my  former 
Masters  student  Jason  J.  Sarich.  Jason  completed  his  MS  in  December  2001,  and  is 
now  employed  by  the  Mathematics  and  Computer  Science  Division  of  Argonne  National 
Laboratory.  The  C  routines  are  described  in  Item  [[4(e) (iii)]]. 

(iii)  We  have  created  a  web  site  (http :  //vwv . uwsp .  edu/math/af  elt/slpinput .  html)  pro¬ 
viding  free  access  to  this  C-routine  package. 
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(c)  Task  4 

The  purpose  of  this  task  was  to  obtain  implementations  of  specific  algorithms  in  our  CPA 
family  and  to  assess  their  practical  performance  relative  to  each  other  as  well  as  relative 
to  existing  codes  for  TSSLP’s.  We  have  developed  a  modular  implementation  of  the  CPA 
family  that  allows  obtaining  specific  members  of  the  CPA  family  as  mentioned  in  Item  (4.1) 
in  [P,  p.  D-15].  We  have  named  our  modular  software  package  CPA.  We  have  also  developed 
a  parallel  version  of  our  code  using  PVM  [11]  that  runs  on  a  network  of  computers  of  the 
Department  of  Mathematics  of  Washington  State  University. 

(i)  Our  attempts  to  obtain  existing  codes  for  TSSLP’s  as  stated  in  [P,  Item  (4.2)]  of  the 
proposal  have  not  been  successful.  Therefore,  while  we  would  continue  to  seek  these 
codes  from  other  researchers,  we  have  developed  modules  within  the  CPA  framework 
that  allow  obtaining  an  implementation  of  the  algorithm  of  Van  Slyke  and  Wets  [22], 
the  most  popular  existing  algorithm  for  stochastic  programming. 

(ii)  Using  our  implementation  of  the  Van  Slyke  and  Wets  algorithm  as  the  representative  of 
the  existing  algorithms  for  stochastic  linear  programs,  we  have  performed  the  compre¬ 
hensive  computational  experiments  as  outlined  in  [P,  Items  (4.2), (4. 3)].  In  particular, 
we  have  performed  experiments  to  compare  different  algorithms  within  CPA,  to  assess 
the  effects  of  parallelization,  to  assess  the  use  of  deep  cuts  relative  to  shallow  cuts,  and 
to  measure  the  dependence  of  performance  on  K.  All  these  experiments  have  been 
performed  using  our  test  problem  collection,  and  our  C-routines  that  convert  input 
for  the  problem  instances  in  SMPS  data  into  the  data  structure  on  which  CPA  was 
developed. 

A  paper  detailing  the  development  of  our  software  package  CPA  and  our  computational 
experience  with  it  will  be  prepared  and  submitted  for  publication  shortly.  At  this  point,  we 
state  the  following  two  general  observations  on  these  computational  results. 

(iii)  As  indicated  in  [P,  §3],  the  three  new  classes  of  algorithms  that  were  evaluated,  namely, 
the  ellipsoid,  the  analytic  center,  and  the  volumetric  center  classes  all  have  complexity 
linear  in  K  under  certain  conditions.  On  the  other  hand,  the  complexity  of  Van  Slyke 
and  Wets  [22]  algorithm  (VWA)  is  exponential.  Thus  one  may  expect  a  threshold 
value  K  such  that  for  all  K  >  K  the  new  algorithms  perform  better  than  VWA.  The 
important  practical  question,  however,  is  whether  K  is  reasonably  low. 

Our  computational  results  indicate  that  for  certain  problem  families  in  our  test  prob¬ 
lem  collection  the  value  of  K  for  the  comparison  of  a  certain  member  of  the  volumetric 
center  class  in  CPA  and  VWA  is  reasonably  low:  this  volumetric  center  algorithm  in 
CPA  performs  better  than  VWA  even  for  moderate  K  values.  In  addition,  when  VWA 
performs  better,  its  performance  is  not  much  better  than  that  of  the  volumetric  center 
algorithm. 

On  the  hand,  the  performance  of  the  ellipsoid  algorithms  and  the  analytic  center  algo¬ 
rithms  in  CPA  were  worse  than  that  of  VWA  in  our  experiments. 

We  were  not  surprised  by  the  poor  practical  performance  of  the  ellipsoid  class:  this 
is  well-known  even  in  the  case  of  deterministic  linear  programs.  However,  we  were 
surprised  by  the  poor  performance  of  the  analytic  center  class,  because  in  the  deter¬ 
ministic  case  interior  point  algorithms  based  on  analytic  center  notions  are  known  to 
perform  better  than  simplex  algorithms  for  certain  classes  of  practical  problems. 

Our  observations  are  important,  because  most  practical  interior  point  algorithms  for 
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deterministic  optimization  are  based  on  analytic  center  notions. 

(iv)  Our  computational  experience  with  the  parallel  version  of  CPA  indicates  that  all  four 
classes  of  algorithms,  i.e.  the  three  new  classes  of  algorithms  in  CPA,  and  VWA, 
parallelize  nearly  linearly.  This  observation  is  important  because  on  some  problem 
instances  with  large  K  the  serial  version  of  the  fastest  algorithm  in  CPA  can  have 
running  times  of  several  hours.  Thus  the  parallel  version  of  CPA  by  virtue  of  its  good 
speed-up  can  reduce  the  running  time  for  these  large  problems  very  significantly. 

(v)  We  have  placed  CPA  on  NEOS  (Network  Enabled  Optimization  Software)  server  of 
Argonne  National  Laboratory.  NEOS  server  provides  a  way  for  users  to  provide  input 
data  for  an  optimization  problem  to  software  available  on  the  server,  have  the  model 
solved  by  the  software,  and  then  to  have  the  solution  reported  back  to  them,  all  via 
the  internet.  See  the  article  [19]  for  details.  NEOS  is  freely  accessible.  Our  software 
CPA  is  available  at  http://www-neos.mcs.anl.gOv/neos/solvers/SLP:CPA/. 

We  now  describe  several  important  results  obtained  in  this  project  that  are  indirectly  related  to 
the  tasks  mentioned  in  [P,  pp.  D-14,15,16].  Therefore,  we  label  these  as  peripheral  results,  but 
their  importance  and  relevance  are  equal  to  those  mentioned  in  Items  [[3(a),  (b),  (c)]]  above. 

(d)  Peripheral  Result  1 

The  field  of  interior  point  algorithms  (this  project  investigated  some  of  them  for  stochastic 
optimization)  began  with  the  seminal  paper  by  Karmarkar  [12]  in  1984,  which  presented  a 
polynomial  algorithm  for  (deterministic)  linear  programs  that  also  had  excellent  practical 
computational  performance.  The  algorithm  in  that  paper  was  derived  using  a  map  referred 
to  as  a  “projective  transformation”.  This  map  appeared  to  be  strange  since  no  other  al¬ 
gorithm  known  until  then  (except  for  a  little  known  algorithm  that  Davidon  [7]  presented 
in  1980)  used  a  map  more  general  than  affine  maps.  Karmarkar  [12]  did  not  provide  any 
reason  for  choosing  this  map. 

In  Item  [[4(a)  (i)]],  we  show  that  the  projective  transformation  of  Karmarkar  [12],  and  the 
map  of  Davidon  [7]  are  the  only  continuous,  injective  maps  that  preserve  convexity  (of  sets) 
in  the  setting  of  each  algorithm.  Preservation  of  convexity  is  a  very  desirable  property  in 
the  design  of  algorithms. 

The  proof  of  the  results  in  Item  [[4(a)  (i)]]  is  based  on  a  new  characterization  of  convexity¬ 
preserving  maps  proved  in  Item  [[4(a)(ii)]].  We  were  surprised  when  pure  mathematicians 
indicated  that  our  characterization  is  new.  Therefore,  Item  [[4(a)  (ii)]]  is  written  in  a  setting 
much  more  general  than  that  required  in  algorithmic  work  as  in  the  present  project  and  in 
Item  [[4(a)  (i)]]. 

The  work  in  Item  [[4(a) (ii)]]  required  a  foundation  for  projective  spaces  not  available  in 
the  pure  mathematics  literature.  The  enthusiasm  we  received  from  the  referees  of  Item 
[[4(a) (ii)]],  prompted  us  to  present  this  foundation  separately  in  Item  [[4(d) (ii)]]. 

This  work  was  performed  in  collaboration  with  Dr.  William  C.  Davidon,  whose  work  in  the 
mid  1950’s  may  arguably  be  considered  as  the  beginning  of  the  field  of  algorithmic  nonlinear 
optimization. 

(e)  Peripheral  Result  2 

The  Peripheral  Result  1  above  dealt  with  characterizing  continuous,  injective  and  convexity¬ 
preserving  maps  on  subsets  of  vector  spaces  as  projective  maps,  and  demonstrating  the 
relationships  of  maps  used  in  the  derivation  of  two  seminal  algorithms  (Karmarkar’s  algo¬ 
rithm  for  linear  programming  [12]  and  Davidon’s  collinear  scaling  algorithms  for  nonlinear 
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optimization  [7])  to  projective  maps. 

In  Item  [[4(d)  (iii)]],  we  have  shown  that  by  using  an  apparently  new  notion  that  we  term 
a  meridian  one  can  quite  simply  describe  one-dimensional  projective  spaces  in  a  geometric 
manner  without  using  the  sophisticated  algebraic  notion  of  a  field.  We  were  pleasantly  sur¬ 
prised  by  the  fact  that  this  apparently  new  result  in  pure  mathematics  was  motivated  by 
our  study  of  computational  optimization  algorithms. 

(f)  Peripheral  Result  3 

As  mentioned  in  Task  [P,  3.2,  p.  D-15],  one  of  the  technical  issues  that  require  attention  in 
the  case  of  volumetric  center  and  analytic  center  classes  of  algorithms  proposed  in  [2]  is  the 
updating  of  iterate  i*  to  Xk+i  once  a  cut  is  added  or  deleted.  This  requires  an  approximate 
minimization  of  certain  potential  functions.  While  Newton’s  method  with  line  searches 
can  be  used  to  perform  this  approximate  minimization  as  suggested  in  [2],  an  attractive 
alternative  was  discussed  in  [1].  In  [I]  certain  line  search  termination  criteria  are  shown  to 
be  valid  for  strongly  convex  functions.  While  the  potential  functions  that  arise  in  stochastic 
linear  programs  satisfy  these  strong  convexity  assumptions,  we  show  in  Item  [[4(a)  (iv)]] 
that  the  line  search  termination  criteria  given  in  [1]  would  be  valid  under  much  milder 
assumptions:  they  are  valid  for  convex  functions  or  for  strictly  pseudo  convex  functions. 
However,  iii  Item  [[4(a)  (iv)]]  we  also  give  a  counterexample  to  show  that  these  line  search 
termination  criteria  may  fail  for  nonconvex  functions. 

(g)  Peripheral  Result  4 

This  continues  the  work  described  in  Peripheral  Result  3.  Specifically,  we  have  developed 
new  algorithms  and  software  for  bound-constrained  nonlinear  optimization  based  on  Davi- 
don’s  collinear  scalings  [7]. 

In  Item  [[4(d) (iv)]],  we  describe  the  derivation  of  these  new  algorithms  and  report  results 
of  our  computational  experiments  with  them.  In  Item  [[4(d)  (v)]],  we  report  a  complete 
convergence  analysis  of  the  new  algorithms. 

We  now  describe  results  obtained  with  work  in  [P,  Tasks  1,5, 6, 7]. 

(h)  Task  1 

The  purpose  of  this  theoretical  task  was  to  extend  the  work  described  in  [P,  §3]  for  the 
linear  case  to  the  quadratic  and  convex  cases  thus  obtaining  ellipsoid,  analytic  center  and 
volumetric  center  classes  of  algorithms  for  TSSQP’s  and  TSSCP’s. 

We  have  derived  ellipsoid,  analytic  center  and  volumetric  center  algorithm  classes  for  TSSQP’s 
and  shown  that  they  have  polynomial  worst  case  complexity  linear  in  K.  Thus  the  results 
in  [P,  §3]  for  the  linear  case  extends  to  the  quadratic  case. 

(i)  Task  5 

The  purpose  of  this  task  is  to  repeat  Tasks  3  and  4  on  TSSLP’s  for  TSSQP’s. 

The  first  step  of  this  task  is  obtaining  the  analog  of  CPA  for  TSSQP’s.  This  work  is  in 
progress. 

(j)  Task  6 

This  task  (and  Task  7)  below  are  concerned  with  low-complexity  algorithms  for  MSSLP’s. 
While  algorithms  for  two-stage  problems  can  be  extended  to  handle  problems  with  more 
than  two  stages,  doing  so  in  a  manner  that  ensures  worst-case  complexity  polynomial  in  the 
number  of  realizations  is  not  at  all  clear.  The  purpose  of  this  task  was  to  derive  algorithms 
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that  belong  to  the  class  of  direct  interior  point  methods  (see  [P,  §§2.1.2,  2.2.2]).  As  indicated 
in  the  statement  of  [P,  Task  6],  the  algorithm  of  Birge  and  Qi  [5]  for  two-stage  problems 
is  a  direct  interior  point  algorithm  with  worst-case  complexity  proportional  to  K1 5  where 
K  is  the  number  of  realizations.  In  the  statement  of  [P,  Task  6]  we  conjectured  that  a 
similar  algorithm  could  be  developed  for  the  multistage  problem  with  worst-case  complexity 
proportional  to  K15.  A  crucial  aspect  of  the  algorithm  of  Birge  and  Qi  [5]  that  results  in 
the  complexity  bound  proportional  to  K 1-5  is  a  procedure  that  computes  a  certain  search 
direction  in  an  efficient  manner.  We  have  extended  that  procedure  to  the  multistage  case. 
This  extension  is  a  major  step  in  arriving  at  an  algorithm  with  complexity  proportional  to 
Kl*  for  MSSLPs. 

(k)  Thsk  7 

This  task  was  concerned  with  decomposition  interior-point  algorithms  for  multistage  prob¬ 
lems.  Note  that  the  algorithms  in  Tasks  1  through  4  are  decomposition  interior-point  al¬ 
gorithms  for  two-stage  problems.  Such  algorithms  for  two-stage  problems  have  complexity 
linear  in  K,  and  due  to  decomposition  they  can  be  implemented  to  exploit  parallel  process¬ 
ing  directly.  As  indicated  in  the  statement  of  Task  7,  the  search  for  such  algorithms  for 
multistage  problem  is  a  difficult  theoretical  task.  We  have  initiated  this  task.  Specifically, 
we  have  begun  to  extend  the  two-stage  algorithm  of  Zhao  [23]  to  the  multistage  case,  using 
volumetric  barriers  instead  of  the  logarithmic  barriers  that  Zhao  [23]  uses.  Note  that  loga¬ 
rithmic  barriers  form  the  basis  for  analytic  centers,  and  our  computational  experience  with 
the  two-stage  case  (see  Item  [[3(c)  (iii)]]  above)  indicates  that  volumetric  center  algorithms 
perform  much  better  in  practice  than  analytic  center  algorithms. 

Yuntao  Zhu,  a  new  doctoral  student  of  the  author,  has  just  begun  exploring  these  ideas  as 
part  of  his  doctoral  dissertation  research. 


4.  Listing  of  all  publications  and  technical  reports  supported  under  this  grant 

Copies  of  all  these  papers  have  been  submitted  to  ARO  according  to  Reporting  Instructions 
(ARO  Form  18)  and  are  available  from  the  author  (email:  ariSwsu.edu).  Manuscripts  sub¬ 

mitted  for  publication  that  have  not  yet  appeared  in  the  open  literature  are  also  available  on  the 
web  at  http: //www. wsu.edu :8080/~ari/preprints.html. 

(a)  Papers  published  in  peer-reviewed  journals 

(i)  K.  A.  Ariyawansa,  W.  C.  Davidon  and  K.  D.  McKennon,  ‘On  a  characterization  of 
convexity-preserving  maps,  Davidon’s  collinear  scalings  and  Karmarkar’s  projective 
transformations’,  Mathematical  Programming  (1)  90  (2001)  153-168. 

(ii)  K.  A.  Ariyawansa,  W.  C.  Davidon  and  K.  D.  McKennon,  ‘A  characterization  of 
convexity-preserving  maps  from  a  subset  of  a  vector  space  into  another  vector  space’ 
Journal  of  the  London  Mathematical  Society  (2)  64  (2001)  179-190. 

(iii)  K.  A.  Ariyawansa  and  A.  J.  Felt,  ‘On  a  new  collection  of  stochastic  linear  programming 
test  problems’,  INFORMS  Journal  on  Computing  (accepted  for  publication  and  in 
press). 

(iv)  K.  A.  Ariyawansa  and  W.  L.  Tabor,  ‘A  note  on  line  search  termination  criteria  for 
collinear  scaling  algorithms’,  Computing  70  (2003),  25-39. 
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(b)  Papers  published  in  non-peer-reviewed  journals  or  in  conference  proceedings 

None. 

(c)  Papers  presented  at  meetings  but  not  published  in  conference  proceedings 

(i)  On  a  characterization  of  convexity-preserving  maps  (17th  International  Symposium  on 
Mathematical  Programming,  Atlanta,  GA,  August  7-11,  2000) 

(ii)  A.  J.  Felt,  K.  A.  Ariyawansa  and  J.  J.  Sarich,  ‘Open  source  input  routine  for  SMPS 
data’,  INFORMS  International  Meeting,  Maui,  Hawaii,  June  17-20,  2001. 

(iii)  A.  J.  Felt  and  K.  A.  Ariyawansa,  ‘A  collection  of  multistage  stochastic  linear  pro¬ 
gramming  test  problems’,  Ninth  International  Conference  on  Stochastic  Programming, 
Berlin,  Germany,  August  25-31,  2001. 

(iv)  K.  A.  Ariyawansa  and  A.  J.  Felt,  ‘Recent  numerical  experience  with  an  implementation 
of  the  volumetric  center  cutting  plane  algorithm  for  stochastic  programming’,  Seventh 
SIAM  Conference  on  Optimization,  Toronto,  Canada,  May  20-22,  2002. 

(v)  K.  A.  Ariyawansa,  A.  J.  Felt,  J.  J.  Sarich  and  R.  O’Fallon,  ‘On  the  computational 
performance  of  a  volumetric  center  algorithm  for  stochastic  programming’,  INFORMS 
National  Meeting,  San  Jose,  CA,  November  17-20,  2002. 

(vi)  K.  A.  Ariyawansa,  ‘New  algorithms  with  polynomial  complexity  for  a  class  of  stochastic 
optimization  problems’,  Presentation  for  the  Board  of  Visitors,  Department  of  Mathe¬ 
matics,  Washington  State  University,  Pullman,  WA  October  3,  2002. 

(vii)  Cristina  Cacho,  K.  A.  Ariyawansa  and  Andrew  J.  Felt,  ‘On  the  construction  of  a  family 
of  stochastic  programming  test  problems’,  INFORMS  National  Meeting,  Atlanta,  GA, 
October  19-22,  2003. 

(d)  Manuscripts  submitted,  but  not  published 

All  these  manuscripts  are  presently  under  consideration  by  the  peer-reviewed  journals  indi¬ 
cated. 

(i)  M.  Vaziri,  K.  A.  Ariyawansa,  K.  Tomsovic  and  A.  Bose,  ‘On  the  accuracy  of  approx¬ 
imating  a  class  of  NP-complete  quadratic  programs  (QP)  by  a  class  of  polynomially 
bounded  linear  programs  (LP)’  (submitted  to  IEEE  Transactions  on  Circuits  and  Sys¬ 
tems). 

(ii)  K.  A.  Ariyawansa,  W.  C.  Davidon  and  K.  D.  McKennon,  ‘A  coordinate-free  foundation 
for  projective  spaces  treating  projective  maps  from  a  subset  of  a  vector  space  into 
another’  (submitted  to  Dissertationes  Mathematicae). 

(iii)  K.  A.  Ariyawansa,  W.  C.  Davidon  and  K.  D.  McKennon,  ‘One-dimensional  projective 
space:  Avatar  of  a  meridian’  (submitted  to  Journal  of  Geometry). 

(iv)  K.  A.  Ariyawansa  and  W.  L.  Tabor,  ‘A  class  of  collinear  scaling  algorithms  for  bound- 
constrained  optimization:  Derivation  and  computational  results’  (submitted  to  Opti¬ 
mization  Methods  and  Software). 

(v)  K.  A.  Ariyawansa  and  W.  L.  Tabor,  ‘A  class  of  collinear  scaling  algorithms  for  bound- 
constrained  optimization:  Convergence  theorems’  (submitted  to  Optimization). 

(e)  Technical  reports  submitted  to  ARO 

This  list  contains  technical  reports  (not  intended  for  journal  publication),  Master’s  project 

reports,  and  doctoral  dissertations. 
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(i)  A  Collection  of  Multistage  Stochastic  Linear  Programming  Test  Problems  (Version 
1),  Technical  Report  00-3,  Department  of  Pure  and  Applied  Mathematics,  Washing¬ 
ton  State  University,  Pullman,  WA  99164-3113  (This  reports  the  present  status  of  an 
evolving  effort;  see  Item  [[3(a)  (iv)]]  above.) 

(ii)  J.  J.  Sarich,  ‘Expanding  the  Capabilities  of  the  TAO  Optimization  Package’,  Group 
Research  Project — M.S.  in  Applied  Mathematics,  Department  of  Mathematics,  Wash¬ 
ington  State  University,  December  2001. 

(iii)  J.  J.  Sarich,  ‘A  C  Implementation  of  an  Input  Routine  for  Two-Stage  Stochastic  Linear 
Programs’,  Individual  Project  Report — M.S.  in  Applied  Mathematics,  Department  of 
Mathematics,  Washington  State  University,  December  2001. 

(iv)  Wayne  L.  Tabor,  ‘Collinear  Scaling  Algorithms  for  Optimization  Using  Line  Searches 
and  Trust  Regions’,  Ph.D.  Dissertation  in  Mathematics,  Department  of  Mathematics, 
Washington  State  University,  Pullman,  WA,  December  2002. 
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Stevens  Point,  Stevens  Point,  WI  (former  doctoral  student  of  the  P.I.  at  Washington  State 
University,  participated  in  this  project  but  was  not  supported  by  this  grant). 

(c)  Jason  J.  Sarich,  Research  Associate,  Mathematics  and  Computer  Science  Division,  Argonne 
National  Laboratory,  Argonne,  IL  (participated  in  this  project,  was  supported  by  this  grant 
as  a  research  assistant,  and  received  the  MS  in  Applied  Mathematics  from  Washington  State 
University  in  December  2001). 

(d)  Wayne  L.  Tabor,  Visiting  Assistant  Professor  of  Mathematics,  Washington  State  University 
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this  grant,  and  received  the  Ph.D.  in  Mathematics  from  Washington  State  University  in 
December  2002). 
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6.  Report  of  inventions 

In  this  section  we  list  the  following  three  “technology  transfer”  items. 

(a)  A  new  stochastic  programming  test  problem  collection  with  free  access  at 

http://www.uwsp.edu/math/afelt/slptestset.html  (see  Item  [[3(a)]]  above). 
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(b)  A  new  C-routine  package  for  converting  SMPS  [4]  input  data  into  data  structures  suitable 
for  implementing  algorithms.  This  package  is  freely  available  at 
http://www.uwsp.edu/math/afelt/slpinput.html  (see  Item  [[3(b)]]  above). 

(c)  A  new  software  package  CPA  for  two-stage  stochastic  programming.  This  package  can  be 
used  freely  via  the  NEOS  [19]  server 

(http://www-neos.mcs.anl.gOv/neos/solvers/SLP:CPA/)  of  the  Argonne  National  Lab¬ 
oratory.  See  Item  [[3(c)]]  above. 
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